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APPARATUS FOR DETECTING AND RECOVERING DATA 
Field of Invention 

The present invention relates to methods and apparatus for detecting and 
recovering data embedded in information material. 
5 Information material as used herein refers to and includes one or more of video 

material, audio material and data material. Video material in this context may be still 
images or moving images. 
Background of Invention 

Steganography is a technical field relating to the embedding of data into 
10 material such as video material, audio material and data material in such a way that the 
data is imperceptible in the material. 

Watermarks £ire data embedded in material such as video material, audio 
material and data material. A watermark may be imperceptible or perceptible in the 
material. 

15 A watermark may be used for various purposes. It is known to use watermarks 

for the purpose of protecting the material against, or trace, infringement of the 
intellectual property rights of the owner(s) of the material. For example a watermark 
may identify the owner of the material. 

Watermarks may be "robust" in that they are difficult to remove from the 

20 material. Robust watermarks are usefiil to trace the provenance of material whicTi is 
processed in some way either in an attempt to remove the mark or to effect legitimate 
processing such as video editing or compression for storage and/or transmission. 
Watermarks may be "fi-agile" in that they are easily damaged by processing which is 
useful to detect attempts to remove the mark or process the material. 

25 . Visible watermarks are useful to allow, for example, a customer to view an 

image via, for example, the Internet to determine whether they wish to buy it but 
without allowing the customer access to the unmarked image they would buy. The 
watermark degrades the image and the mark is preferably not removable by the 
customer. Visible watermarks are also used to determine the provenance of the 

30 material into which they are embedded. 
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In US patent 5, 930, 369 (Cox et al) it hcis been proposed to embed data into 
material such as images to form a watermark by converting the material into the 
transform domain and adding the data to the image in the transform domain. For the 
example of images and the Discrete Wavelet Transform of these images into the 
transform domain, the data to be added can be combined with the wavelet coefficients 
of one of a plurality of sub-bands which are formed in the transform domain. 
Generally, the data to be embedded is arranged to modulate a predetermined data 
sequence such as a Pseudo Random Bit Sequence (PRBS). For example, each bit of 
the data to be embedded is arranged to modulate a copy of the PRBS, and this copy is 
then added, for example into one of the sub-bands of the image in the transform 
domain. The image is then converted back to the spatial domain. 

If it is desired to detect and recover the embedded data from the image, the 
image is converted back to the tremsform domain and the embedded data is recovered 
from the sub-band in the transform domain by cross-correlating the transform 
coefficients in the sub-band with the predetermined data sequence which is known to 
the detecting apparatus. Data sequences such as a PRBS have properties which 
provide a high auto-correlation result when the sequence coincides with itself and a 
low or ideally zero output at all other shifts of the sequence with respect to itself As 
such, the embedded data can be recovered from the result of correlating the sub-band 
data with the predetermined data sequence. 
Summary of Invention 

According to the present invention there is provided an apparatus for detecting 
and recovering data embedded in information material, the data having been embedded 
in the information material by modulating a predetermined data sequence with the data 
to be embedded to form modulated data and combining the modulated data with the 
information material, the apparatus comprising a correlation processor operable in 
combination v^th a data sequence processor to form a correlation sequence, and a data 
processor operable under control of the correlation processor to correlate information 
material data symbols, to which the modulated data have been combined, with the 
correlation sequence, to form a correlation output signal representing the correlation 
between the information material data symbols 2ind the correlation data sequence, and 
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to recover the embedded data from the correlation output signal, wherein the 
correlation sequence comprises a plurality of predetermined data sequence versions, 
each version representing a possible form of the predetermined data sequence for 
recovering the embedded data symbols from the information material. 

Data representing information material such as images are often processed in 
order to convert the data from one form to another, so that, for example the images c£in 
be compression encoded and decoded. A fiirther example in which images may be 
processed is if the images are convert into another form using transform processing. 
Such transforms may be for example the Discrete Wavelet Transform or the Discrete 
Fourier or Cosine Transform, A result of processing the material, can cause the data 
representing the material to be shifted, in particular in the transform domain. 

If it is desired to recover the embedded data by correlating the modulated data 
from the transform domain with the predetermined data sequence known to a decoding 
data processor, it may not be possible to detect the embedded data because the 
predetermined data sequence will no longer align with the modulated version of the 
predetermined data sequence, because the data in the transform domain has shifted. 
Effectively, the shift of the data in the sub-band will have an effect of introducing a 
corresponding relative shift between the correlating predetermined data sequence and 
the modulated version of the data sequence in the sub-band. The correlation output 
signal may be low or zero, thereby preventing the data from being correctly detected. 

Similarly, the predetermined data sequence which is modulated by the data to 
be embedded and combined with the information material may be one of a set of 
predetermined data sequences. The predetermined data sequence used to embed the 
data in the information material may be unknown at the decoder. 

Embodiments of the present invention provide an improvement in the 
likelihood of correctly detecting the embedded data by generating a correlation 
sequence comprising a plurality of predetermined data sequence versions. Each 
version may be the same predetermined data sequence but shifted with respect to one 
another. Each shift provides a possible relative shift of the material data in the 
transform domain at which there would be no shift between this version and the 
modulated version of the data sequence in the transform domain. A result of the 
correlation will produced a high output signal value, which when compared with a 
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predetermined threshold provides an indication of the embedded data symbol. All 
other versions of the predetermined data sequence in the correlation data sequence will 
produce a low correlation output or zero, and so will not interfere with the detection of 
the embedded data symbol. This is because predetermined data sequences such as 
5 PRBSs have such correlation properties that the other shifted versions of the PRJ3S 
will represent noise with respect to the correlation wdth the wanted signal. Therefore 
provided the correlation output signal is high enough in the presence of the other 
shifted versions the embedded data symbol can be detected firom the transform domain 
data in spite of any shifts of the transform domain data. Generally, it is not necessary 

10 to know which of the shifted versions in the correlation data sequence aligns with .the 
predetermined data sequence of the embedded data. 

In preferred embodiments, the number of versions of the predetermined data 
sequence may be equal to a number of possible relative shifts of the modulated 
predetermined data sequence, each of the plurality of predetermined data sequences 

1 5 being shifted with respect to each other, each shift representing a number of symbols 
by which the modulated data symbols may have shifted in the transform domain. 

In altemative embodiments, in which the predetermined data sequence used to 
form the modulated data is one of a set of predetermined data sequences, each of said 
predetermined data sequence versions of the correlation data sequence is a different 

20 predetermined data sequence of the set of possible predetermined data sequences 
which may have been used to form the modulated data. As will be appreciated, the 
wanted predetermined data sequence which corresponds to the predetermined data 
sequence used to be embed the data v/ill produce a correlation result fi-om which the 
embedded data can be recovered, whereas the other sequences will form noise. The 

25 embedded data may be recovered therefore, without knowing which of the 
predetermined data sequences was used to embed the data. 

In preferred embodiments, the correlation sequence may be formed by 
selecting versions of the predetermined data sequence and reversing the polarity of the 
selected versions of the predetermined data sequence. It has been found that by 

30 reversing the polarity of selected versions of the predetermined data sequence, for 
example, reversing the polarity of alternate predetermined data sequence versions a 
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reduction of the interference of the unwanted versions is provided with respect to the 
wanted version of the predetermined data sequence. 

As will be appreciated from the foregoing description, in preferred 
embodiments, the predetermined data sequence is a PRES. The correlation sequence 
5 therefore comprises a plurality of versions of the PRBS each being shifted with respect 
to each other. 

In preferred embodiments the data may be combined with the information 
material in a transform domain representation of the information material. As a result, 
the effect of for example a data symbol combined with a transform domain data 

10 symbol may be spread over several information material data symbols, thereby 
reducing any perceivable effect on the information material. Any appropriate 
transform may be used. However, in preferred embodiments, the transform may be the 
discrete wavelet transform, the data symbols in the transform domain being divided 
into each of a plurality of sub-bands comprising wavelet coefficients, the data being 

1 5 added to at least one of the sub-bands. 

Although the information material in which the data has been embedded can be 
any form or type of material, in preferred embodiments the information material is 
video, audio or audio/video material. 

Various further aspects and features of the present invention are defined in the 

20 appended claims. 

Brief Description of the Drawings 

Figure 1 is a schematic block diagram of a watermarking system; 
Figure 2 is a schematic block diagram of a watermark embedder appearing in 
Figure 1; 

25 Figure 3 is a schematic representation illustrating the form of the wavelet 

domain transform; 

Figure 4 is a schematic representation of an iterative filtering process which 
can be used to form the wavelet traiisform; 

Figure 5 is a schematic block diagram of a combiner forming part of the 
30 watermark embedder shown in Figure 2; 



P/10149GB 6 

Figure 6 provides an illustrative representation of a transform domain image in 
v^hich data is added by the combiner shown in Figure 5: 

Figure 7 is a schematic block diagram of a watermark decoder appearing in 
Figure 1: 

Figure 8 is a schematic block diagram of a correlator forming part of the 
watermark decoder shown in Figure 7 

Figure 9 is a schematic representation illustrating the formation of a correlation 
data sequence used by the correlator of Figure 8; 

Figure 10 provides an illustrative representation of a trgmsform domain image 
from which embedded data is to be detected and recovered; 

Figure 11 is a schematic representation illustrating the operation of a data 
processor shown in Figure 8 performing a correlation of the correlation data sequence 
of Figure 9 and the transform domain data of Figure 10; 

Figure 12 is a schematic representation illustrating the formation of a further 
correlation data sequence which may be used by the correlator of Figure 8; 

Figure 13 is a schematic representation illustrating the operation of a data 
processor shown in Figure 8 performing a cross-correlation of the correlation data 
sequence of Figure 12 and the transform domain data of Figure 10; 

Figures 14A and 14B are schematic block diagrams of the structure of an 
extended and a basic UMID respectively. 
Description of Preferred Embodiments 

An example embodiment of the present invention will be described with 
reference to a watermarking system in which data is embedded into a video image. 
Any type of data can be embedded into the image. However, advantageously the data 
embedded into the image may be meta data which describes the image or identifies 
some attributes of the content of the image itself. An example of meta data is the 
Universal Material Identifier (UMID). A proposed structure for the UMID is disclosed 
in SMPTE Journal March 2000. A more detailed explanation of the structure of the 
UMID will be described later. 
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Watermarking System 

Figure 1 illustrates a watermarking system, generally 10, for embedding a 
watermark into a video image 115, and recovering and removing a watermark from the 
video image 115. The watermarking system 10 in figure 1 comprises an image 
5 processor 100 for embedding the watermark into the video image, and a decoding 
image processor 102 for detecting and recovering the watermark, and for removing or 
'washing' the watermark from the video image. 

The image processor 100 for embedding the watermark into the video image 
comprises a strength adapter 180, and a watermark embedder 120. The watermark 

10 embedder 120 is arranged to embed the watermark into the video image 115, produced 
from the source 110, to form a watermarked image 125, The watermark to be 
embedded into the video image is formed from data 175 representing a UMID. 
Generally, the UMID identifies the content of the video image, although it will be 
appreciated that other types of meta data which identify the content or other attributes 

15 of the image can be used to form the watermark. In preferred embodiments the 
watermark embedder 120 embeds the UMID into the video image 115 in accordance 
with a particular application strength 185 from the strength adapter 180. The strength 
adapter 180 determines the magnitude of the watermark in relation to the video image 
115, the application strength being determined such that the watermark may be 

20 recovered whilst minimising any effects which may be perceivable to a viewer of the 
watermarked image 125. After embedding the watermark, the image may be 
transmitted, stored or further processed in some way, such as for example, 
compression encoding the image. This subsequent processing and transmitting is 
represented generally in Figure 1 as line 122. 

25 In Figure 1 the decoding image processor 102 for detecting and removing the 

watermark is shown as comprising a watermark decoder 140, a data store 150 and a 
watermark washer. 130 which removes the watermark from the watermarked image 
125. 

The watermark decoder 140 detects the watermark from the watermarked video 
30 image and in the present example embodiment, generates a restored UMID 145 from 
the watermarked image 125. The watermark washer 130 generates a restored image 
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135, by removing as far as possible the watermark from the watermarked image 125. 
In some embodiments, the watermark washer 130 is operable to remove the watermark 
from the image substantially without leaving a trace. The restored image 125 may 
then be stored in a store 150, transmitted or routed for further processing. 
The Watermark Embedder 

The watermark embedder will now be described in more detail with reference 
to Figure 2, where parts also appearing in Figure 1 have the same numerical 
references. In Figure 2 the watermark embedder 120 comprises a pseudo-random 
sequence generator 220, an error correction encoder 200, a wavelet transformer 210, 
an inverse wavelet transformer 250, a modulator 230 and a combiner 240. 

The error correction encoder 200 receives the UMID 175 and generates an 
error correction encoded UMID comprising redundant data in combination with the 
UMID, in accordance with an error correction encoding scheme. It will be appreciated 
that various error correction coding schemes could be used to encode the UMID. For 
the example embodiment the error correction encoder 200 uses a Bose-Chaudhuri- 
Hocquenghem (BCH) systematic code providing 511 bit code words comprising 248 
source bits of the UMID and 263 bits of redundant parity bits. 

As shown in Figure 2 the error correction encoded UMID 205 is received at a 
first input to the modulator 230. The pseudo-random sequence generator 220 outputs a 
PRBS 225 which is received at a second input to the modulator 230. The modulator 
230 is operable to modulate each copy of a PRBS, generated by the pseudo-random 
sequence generator 220, with each bit of the error correction encoded UMID. In 
preferred embodiments the data is modulated by representing the values of each bit of 
the PRBS in bipolar form (T as +1, and '0' as -1) and then reversing the polarity of 
each bit of the PRBS, if the corresponding bit of the encoded UMID is a '0' and not 
reversing the polarity if the corresponding bit is a T. The modulated PRBS is then 
received at a first input of the combiner 240. The combiner 240 receives at a second 
input the image in which the PRBS modulated data is to be embedded. However the 
data is combined with the image in the transform domain. 

The use of a pseudo-random sequence 225 to generate the spread spectrum 
signal representing the watermark data allows a reduction to be made in the strength of 
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the data to be embedded in the image. By cross-correlating the data in the transform 
domain image to which the modulated PRBS has been added, a correlation output 
signal is produced with a so called correlation coding gain which allows the modulated 
data bit to be detected and determined. As such, the strength of the data added to the 
5 image can be reduced, thereby reducing any perceivable effect on the spatial domain 
image. The use of a spread spectrum signal also provides an inherent improvement in 
robustness of the image because the data is spread across a larger number of transform 
domain data symbols. 

As shown in Figure 2, the wavelet transformer 210 receives the video image 

10 115 from the source 110 and outputs a wavelet image 215 to the combiner 240. The 
image is thus converted from the spatial to the transform domain. The combiner 240 is 
operable to add the PRBS modulated data to the image in the transform domain, in 
accordance with the application strength, provided by the strength adapter 180. The 
watermarked wavelet image 245 is then transformed into the spatial domain by the 

15 inverse wavelet transformer 250 to produce the watermarked image 125. The 
operation of the combiner 240 will be explained in more detail shortly. 

In order to explain the embodiment of the present invention a brief description 
of wavelet transforms will be provided in the following paragraphs with reference to 
Figures 3 and 4. 

20 The Wavelet Transform 

Wavelets are well known and are described in for example "A Really Friendly 
Guide to Wavelets" by C Valens, 1999 rc.valens@.mmdless.comy Valens shows that 
the discrete wavelet transform can be implemented as an iterated filter bank, as used in 
sub-band coding, with scaling of the image by a factor of 2 at each iteration. 

25 Thus, referring to Figure 3 a spatial domain image is applied to a set of high 

pass HP and low pass LP filters. At level 1, the fu-st stage of filtering, the image is 
filtered horizontally and vertically and, in each direction, scaled down by a factor of 2. 
In level 2, the low pass image from level 1 is filtered and scaled in the same way as in 
level 1 . The filtering and scaling may be repeated in subsequent levels 3 onwards. 

30 The result is shown schematically in Figure 4. Figure 4 is a representation 

normal in the art. The horizontal axis H indicates increasing frequency of the sub- 
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bands. At level one the image is spatially filtered into four bands; the lower horizontal 
and vertical band, IHi, IVi; the upper horizontal band hHi, IVi; the upper vertical band 
IHu hVi; and the upper horizontal and vertical band, hHu hVi. At level 2, the lower 
horizontal and vertical band, IHi, IV i is filtered and scaled into the lower horizontal 
and vertical band, IH2, IV2; the upper horizontal band hH2, IV2; the upper vertical band 
IH2, hV2; and the upper horizontal and vertical band, hH2, hV2. At level 3 (not shown 
in Figure 4), the lower horizontal and vertical band, IH2, IV2 is further filtered and 
scaled. 
Combiner 

The operation of the combiner 240 will now be explained in more detail. The 
combiner 240 receives the wavelet image 215 from the wavelet transformer 210, and 
the modulated PRBS from the modulator 230 and the application strength 185 from the 
strength adapter 180. The combiner 240 embeds the watermark 235 onto the wavelet 
image 215, by adding, for each bit of the modulated PRBS a factor a scaled by ±1, in 
dependence upon the value of the bit. Selected parts of the wavelet image 215 are 
used to embed the watermark 235. Each pixel of the predetermined region of the 
wavelet image 215 is encoded according to the following equation: 

X\=X,^a„W„ (1) 

Where Xi is the i-th wavelet coefficient, is the strength for the n-th PRBS 

and Wy^ is the «-th bit of the watermark to be embedded in bipolar form, which foi^the 

example embodiment is the bit of the error correction encoded UMID. 

An example of the combiner and the operation of the combiner will now be 
described with reference to Figures 5 and 6. In Figure 5 the combiner 240 is shovm to 
receive the transform domain image from the connecting channel 215 which provides 
the transform domain image to a frame store 236. The frame store 236 is arranged to 
store a frame of transform domain data. The combiner 240 is also arranged to receive 
the spread spectrum encoded and error correction encoded UMID after it has been 
spread using the PRBS (modulated PRBS data). For this example embodiment one 
UMID in this error correction and spread spectrum encoded form is to be embedded in 
the frame of image data vAthin the frame store 236. Thus, each encoded UMID forms 
an item of data which is to be embedded into each frame of image data. To this end. 
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the frame store stores a frame of data representing the image in the wavelet transform 
domain. The data to be embedded is received at a combining processor 237 which 
combines the data to be embedded into selected parts of the wavelet transform domain 
image stored in the frame store 236. The combiner 240 is also provided with a control 
5 processor 238 which is coupled to the combining processor 237. 

In Figure 6 an illustrative representation of a first order wavelet transform is 
presented. This wavelet transform is representative of a frame of the image 
transformed into the wavelet domain and stored in the frame store 236. The wavelet 
transform image WT_IMG is shown to comprise four wavelet domains representative 

10 of sub-bands into which the image has been divided. The wavelets have been labelled 
in correspondence with the illustration provided in Figure 3. In the example 
embodiment of the present invention, the data to be embedded is only written into the 
low vertical, high horizontal frequencies sub-band hHilV^ and the low horizontal, 
high vertical frequencies sub-bands labelled IHihVi. The data is added to these sub- 

15 bands hHilVj, IHihVi by the combining processor 237 under control of the control 

processor 238 as illustrated in Figure 6, according to equation (1). 

By embedding the data in only the two sub-bands hHilVi, IHihVi, the 
likelihood of detecting the embedded data is improved whilst the effects that the 
embedded data will have on the resulting image are reduced. This is because the 
20 wavelet coefficients of the high horizontal, high vertical frequencies sub-bands 
hHjhVi are more likely to disturbed, by for example compression encoding. 

Compression encoding processes such as JPEG (Joint Photographic Experts Group) 
operate to compression encode images by reducing the high frequency components of 
the image. Therefore, writing the data into this sub-band hHihVi would reduce the 
25 likelihood of being able to recover the embedded data. Conversely, data is also not 
written into the low vertical, low horizontal frequencies sub-band IHilVj. This is 
because the human eye is more sensitive to the low frequency components of the 
image. Therefore, writing the data in the low vertical, low horizontal frequencies sub- 
band would have a more disturbing effect on the image. As a compromise the data is 
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added into the high horizontal, low vertical frequencies sub-band hHilVi and the low 
horizontal, high vertical frequencies sub-bands IH^hVi. 

As already explained, an effect of processing the image in the spatial or the 
transform domains is to shift the wavelet coefficients in the sub-band with an effect 
5 that the relative position of the wavelet coefficients within the sub-bands differ from 
their original positions. This is illustrated in Figure 6, by the double headed arrows 
SHFT. Since the processing which may be performed on the image is unknown, the 
amount of the shift of the sub-band coefficients will be unknown. As a result, the 
modulated version of the PRBSs added to the wavelet coefficients of the sub-band will 

10 no longer align, and will have been shifted with respect to a PRBS generated by a 
detector, which is expecting the modulated PRBS versions to be at the same position 
as they were when the modulated PRBSs were added to the sub-band by the data 
embedder. As a result the cross-correlation of wavelet coefficients with the 
reproduced PRBS will produce a low or zero value from which the embedded data is 

1 5 unlikely to be detectable. 

Decoder 

The operation of the watermark decoder 140 in the decoding image processor 
will now be explained in more detail, with reference to Figure 7, where parts also 
20 appearing in Figure 1 bear identical reference numerals. The watermark decoder J 40 
receives the watermarked image 125 and outputs a restored version of the UMID 145. 
The watermark decoder 140 comprises a wavelet transformer 310, a pseudo-random 
sequence generator 320, a correlator 330, a selector 340 and an error correction 
decoder 350. 

25 The wavelet transformer 310 converts the watermarked image 125 into the 

transform domain so that the watermark data can be recovered. The wavelet 
coefficients to which the PRBS modulated data were added by the combiner 240 are 
then read from the two wavelet sub-bands hHilVi, IHihVi in the reverse direction to 
the direction in which the data was added in the combiner 240. These wavelet 

30 coefficients are then correlated with respect to the corresponding PRBS used in the 
watermark embedder. Generally, this correlation is expressed as equation (2), below. 
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where Xy^ is the w-th wavelet coefficient and 7?^ is the R-Xh bit of the PRBS generated 
by the Pseudo Random Sequence Generator 320. 

Q=l;^„.,i?, (2) 

The relative sign of the result of the correlation Cyi then gives an indication of 

5 the value of the bit of the enibed data in correspondence with the sign used to represent 
this bit in the watermark embedder. The data bits recovered in this way represent the 
error correction encoded UMID which is subsequently decoded by the error correction 
decoder 350 using a decoding algorithm for the error correction code used by the 
encoder 200. Having recovered the UMID, the watermark can be removed from the 

10 video image by the watermark washer 130, by performing the reverse of the operations 
performed by the embedder. The correlator 330 appearing in Figure 7 is shown in 
more detail in Figure 8, wherein parts also appearing in Figure 7 bear identical 
numerical references, 

In Figure 8 the correlator 330 is shown to comprise a frame store 33 1 which is 

15 arranged to receive the wavelet transformed image via the connecting channel 315. 
The frame store 33 1 is arranged to store a frame of wavelet transformed image. A data 
sequence processor 332 is arranged to receive the PRBS from the PRBS generator 320 
and to feed this PRBS in sync with a timing signal received from a correlation 
processor 333, via timing channel 336. The correlation processor 333 controls the 

20 correlation of the data symbols (wavelet coefficients) read frorn the sub-band of^the 
image in the frame store 331 with the PRBS. However, to recover the embedded data 
the reproduced PRBS should be aligned with the corresponding modulated version of 
the PRBS added to respective wavelet coefficients from the sub-band. The correlation 
is performed by the data processor 334, under control from the correlation processor 

25 333, using control channel 337, and the data sequence processor 332. 
First Embodiment 

Embodiments of the present invention address a technical problem of 
recovering the embedded data from the wavelet coefficients if there has been a shift in 
the wavelet coefficients in the sub-band. As already explained such a shift can occur if 
30 the image has been processed and can occur when the image is transformed from the 
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spatial domain to the transform domain. If the wavelet transform coefficients have 
shifted within the sub-band, then the PRBS will no longer be aligned with the version 
of the PRBS which has been added to the wavelet coefficients within the sub-band. As 
a result, the correlation with the reproduced PRBS will not provide a correlation output 
signal from which the embedded data symbols are detectable. This is addressed by 
embodiments of the present invention by generating a comjposite sequence hereinafter 
referred to as a correlation data sequence and correlating this with respect to the data 
within the sub-band. The correlation data sequence is formed as illustrated in Figure 
9. 

As illustrated in Figure 9 a number of copies of the PRBS used to embed the 
data in the transform domain are generated by the PRBS generator 320 and fed via the 
connecting channel 325 to the data sequence processor 332, The number of PRBS 
copies generated corresponds to the number of possible shifts of the data within the 
sub-band which can be tolerated. For each of these tolerated shifts, the embedded data 
may still be recovered. As illustrated in Figure 9 the possible integer shifts INT_SHFT 
are shown alongside a corresponding version of the PRBS which has been shifted with 
respect to a central PRBS CENT_P. Effectively, therefore for the illustrative 
embodiment shown in Figure 9, a shift of ±2 symbols can be tolerated and the 
embedded data symbols can still be detected from the output of the correlation signal. 
The five versions of the PRBS at each of -2, -1, 0, +1, +2 shifts are then combined 
together to form a composite data sequence which forms the correlation data sequence 
CORR_SEQ. It is the correlation data sequence CORR_SEQ which is supplied by the 
data sequence processor 332 to the data processor 334 under control of the correlation 
processor 333 to generate the correlation output from which the embedded data 
symbols can be recovered. 

In Figure 10 the data embedded within the high horizontal, low vertical 
frequencies sub-band hHilVi is represented as a sequence of boxes P_SEQ 

representing the continuous regions for which a modulated PRBS has been added to 
the wavelet coefficients. The correlation processor 333 is arranged to read the wavelet 
coefficients sequentially in correspondence with the order in which they were added to 
the sub-band starting with a start position ST_P from which the new PRBS is assumed 
to start. An arrow Rl represents the order of reading the wavelet coefficients which is 
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performed by the correlation processor 333. The data processor 334 effectively then 
calculates the correlation between this set of wavelet coefficients WCT and the 
correlation sequence CORR_SEQ as represented schematically in Figure 11, As a 
result, although the wavelet coefficients within the sub-band may have been shifted 
5 from their expected position, by up to ±2 symbols, then the wavelet coefficients 
recovered will nevertheless align with one of the versions of the PRBS which was used 
to construct the correlation data sequence CORR_SEQ. As a result the data processor 
334 will generate an output correlation signal for the alignment of the wavelet 
coefficients with the corresponding PRBS within the correlation sequence to produce a 

10 signal from which the embedded data symbol can be detected. It is a property of data 
sequences such as PRBSs that other shifts of the PRBS will effectively appear as noise 
with respect to the aligned version of the PRBS which has been added to the wavelet 
coefficients. Since the correlation output will produce a high peak for the aligned 
PRBS, the embedded data symbol will still be detectable from the correlation output 

15 signal in spite of the presence of the other shifted versions of the PRBS, which 
effectively form noise. 
Second Embodiment 

A further embodiment of the present invention is illustrated in Figures 12 and 
13 respectively. The embedded data may have been combined with one of a plurality 

20 of possible PRBSs which may not be known at the watermark decoder processor 140. 
What would be knovm is that the PRBS used to embed the data is one of a limited set 
of PRBSs. In this example embodiment, the correlation sequence is formed by adding 
together the PRBSs corresponding to the set of PRBSs which could have been used to 
spread spectrum encode the embedded data added to the wavelet coefficients. To this 

25 end. Figure 12 illustrates such an arrangement in which the set is considered to 

comprise four PRBSs PI, P2, P3, P4 which are added together to form the correlation 
sequence CORR_SEQ^ The correlation processor 333 then operates in exactly the 
same way as already explained for the first embodiment, but now correlates the 
recovered wavelet coefficients with the composite data sequence CORR__SEQ^ formed 

30 from combining the four different PRBSs. This is illustrated in Figure 13. Again as 
with the embodiment described with reference to Figures 9 and 1 1, the PRBS which 
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has been used to spread spectrum encode the embedded data will correlate with an 
aligned version of the modulated PRBS added to the wavelet coefficients and generate 
a correlation output signal from which .the data processor 334 can detect the embedded 
data symbol. As before the other unwanted PRBSs which were not used to encode the 
5 embedded data symbols will effectively form noise to the correct PRBS. However as 
already explained, a property of data sequences such as PRBSs is that the correlation 
output for the correct PRBS, will be high enough in the presence of the other incorrect 
PRBSs acting as noise that the data symbol can be detectable from the resulting 
correlation output signal. 

10 In preferred embodiments, the correlation processor 334 may be operable to 

form the correlation sequence in the embodiments described herein, by selecting a 
' plurality of the shifted versions of the PRBS and reversing the polarity of the selected 
versions. By reversing the polarity of selected versions of the PRBSs to form the 
correlation data sequence, a reduction of the interference of the unwanted PRBSs to 

15 the wanted PRBS can be provided, when the correlation is performed. For example, 
altemate PRBSs of the correlation sequence can be combined with the polarity of 
alternate PRBSs versions reversed. Since the polarity of the PRBS which forms the 
wanted PRBS will be unknown as a result of the polarity inversions, a corresponding 
adjustment must be made in the decoder in order to interpret the recovered data 

20 correctly. This is because the embedded data recovered from the image may be 

inverted, in that each '0' will appear as a ' 1' and each ' 1 ' will appear as a '0'. In _ 
preferred embodiments, however, at least one symbol of predetermined polarity may 
be embedded into the image. This can be used to adjust the bit values recovered. This 
can be arranged m the correlator 330 itself, or after the error correction decoder. An 

25 advantage of applying correction of the recovered bits after error correction decoding 
is that the error correction code can be used to provide protection for the 
predetermined polarity symbol. Thus after decoding, the value of the predetermined 
polarity symbol is determined and compared vnth the value, which is knovra to the 
decoder. Appropriate correction can then be applied. 

30 Other Embodiments 
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A further improvement can be provided to the embodiments described herein, 
to the effect of identifying which PRBS version from the plurality of PRBS versions 
from which the correlation sequence is formed should be used to recover the • 
embedded data. To this end the data sequence processor 334 operates under the 
5 control of the correlation processor 333 to divide each of the versions of the PRBSs 
into two or more groups and to form correspondingly two or more correlation 
• sequences by combining the respective PRBS versions of the groups. The correlation 
sequences are then correlated separately with the wavelet coefficients. The correlation 
sequence which produces a correlation output signal v/hich exceeds the correlation 

10 threshold is identified. The identified correlation sequence therefore contains the 
wanted PRBS, the wanted PRBS being that which correctly corresponds with the 
version used to form the modulated data. This wanted PRBS can be used subsequently 
to recover the data by performing subsequent correlations. The PRBS versions from 
which the identified correlation sequence is comprised are then divided further into 

1 5 groups from which further correlation sequences are formed. The process is repeated 
to identify which of the correlation sequences contains the v/anted PRBS. After 
further divisions, the wanted PRBS may be unambiguously identified. 
Correspondingly, the identified PRBS provides an indication of which of the shifted 
versions of the PRBS^or which of the different PRBSs is present in the watermarked 

20 image. The identified PRBS can be used therefore for subsequent correlations without 
the interference caused by the unwanted PRBSs. 

In further embodiments, a further correlation sequence could be formed by 
combining together a shifted version of each of the PRBSs PI, P2, P3, P4 to the effect 
that the number added together will correspond to the number of possible shifts of the 

25 wavelet coefficients within the sub-band. As such, this further embodiment will 
provide a facility for recovering the embedded data if the PRBS from a set of PRBSs is 
not known and if the wavelet coefficients v^thin the sub-band have been shifted as a 
result of processing. 

The Universal Material Identifier (UMID) 



30 A brief explanation will now be given of the structure of the UMID, with 

reference to Figure 14A and 14B. The UMID is described in SMPTE Journal March 
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2000. Referring to Figures 14A an extended UMID is shown to comprise a first set of 
32 bytes of a basic UMID, shown in Figure 14B and a second set of 32 bytes referred 
to as signature metadata. Thus the first set of 32 bytes of the extended UMID is the 
basic UMID. The components are: 

•A 12-byte Universal Label to identify this as a SMPTE UMID. It defines the 
tj^e of material which the UMID identifies and also defmes the methods by which the 
globally unique Material and locally unique Instance numbers are created. 

•A l-b)^e length value to define the length of the remaining part of the UMID. 

•A 3 -byte Instance number which is used to distinguish between different 
'instances' of material with the same Material number. 

•A 16-byte Material number which is used to identify each clip. Each Material 
number is the same for related instances of the same material. 

The second set of 32 b3rtes of the signature meta data as a set of packed meta 
data items used to create an extended UMID. The extended UMID comprises the 
basic UMID followed immediately by signature meta data which comprises: 

©An S'hytQ time/date code identifying the time and date of the Content Unit 
creation. 

•A 12-byte value which defines the spatial co-ordinates at the time of Content 
Unit creation. 

•3 groups of 4-byte codes which register the country, organisation and user 

codes. 

More explanation of the UMID structure is provided in co-pending UK patent 
application number 0008432.7. 

Various modifications may be made to the embodiments herein before 
described without departing from the scope of the present invention. Although in this 
example embodiment, the data to be embedded is added to the image in the transform 
domain, in altemative embodiments the data could be represented in the transform 
domain, inverse transformed into the spatial domain, and added to the data in the 
transform domain. 
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CLAIMS 

1 . An apparatus for detecting and recovering data embedded in 
information material, said data having been embedded in said information material by 
5 modulating a predetermined data sequence with the data to be embedded to form 
modulated data and combining said modulated data with said information material, 
said apparatus comprising 

a correlation processor operable in combination with a data sequence processor 
to form a correlation sequence, £md 

10 a data processor operable under control of the correlation processor to correlate 

information material data symbols, with which said modulated data have been 
combined, with said correlation sequence, to form a correlation output signal 
representing the correlation between the information material data symbols and said 
correlation data sequence, and to recover said embedded data from said correlation 

1 5 output signal, wherein said correlation sequence comprises a plurality of 

predetermined data sequence versions, each version representing a possible form of the 
predetermined data sequence for recovering said embedded data symbols from said 
information material. 

20 2. An apparatus as claimed in Claim 1, wherein each of said versions of 

said predetermined data sequence are the predetermined data sequence used to form 
said modulated data shifted with respect to one another. 

3. An apparatus as claimed in Claim 2, wherein the number of said 
25 plurality of versions of said predetermined data sequences is equal to a number of 
possible relative shifts of the information material data symbols to which the 
modulated data have been added, each of said plurality of predetermined data 
sequences being shifted with respect to each other, each shift representing a number of 
symbols by which the information material data sjonbols may have been shifted. 
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4. An apparatus as claimed in Claim 1, wherein each of said 
predetermined data sequence versions is a different predetermined data sequence of a 
set of possible predetermined data sequences which may have been used to form said 
modulated data. 

5. An apparatus as claimed in any preceding Claim, wherein said 
predetermined data sequence is a Pseudo Random Symbol or Bit Sequence. 

6. An apparatus as claimed in any preceding Claim, wherein said 
correlation processor is operable in combination with said data sequence processor, 

to divide each of said predetermined data sequence versions into two or more 

groups, 

to combine the predetermined data sequence version from each group to form a 
corresponding plurality of correlation data sequences, said data processor being 
arranged to correlate said information material data symbols with which said 
modulated data have been combined, with each of said correlation sequences, to form, 
for each, a correlation output signal, and said correlation processor is operable to 
identify which of said groups of predetermined data sequence versions includes the 
predetermined data sequence version for recovering said embedded data symbols from 
the corresponding correlation output signal. 

7. An apparatus as claimed in Claim 6, wherein said correlation processor 
is operable to compare said correlation output signal with a threshold and to identify 
which of said groups includes the predetermined data sequence version for recovering 
said embedded data from the comparison. 

8. An apparatus as claimed in Claims 6 or 7, wherein said correlation 
processor is operable to divide said identified group into further groups, each of said 
groups being combined to form further correlation sequences and to identify which of 
said further groups includes said predetermined data sequence for recovering said 
embedded data, said dividing and said identifying being repeated until the 
predetermined data sequence version for recovering said embedded data is identified. 
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9. An apparatus as claimed in any preceding Claim, wherein said 
correlation sequence is formed by selecting versions of said predetermined data 
sequence and reversing the polarity of said selected versions of said predetermined 

5 data sequence before combining said predetermined data sequence versions to form 
said correlation sequence, 

10. An apparatus as claimed in any preceding Claim, wherein said data is 
combined with said information material by forming a transform domain 

10 representation of said data and combining said data with said information material in 
either said transform domain or the inverse transform domain, said apparatus 
comprising 

a transform processor operable to generate a transform domain representation 
of said information material in which said data has been embedded, said correlation 
1 5 processor being operable to recover said embedded data symbols by correlating 

transform domain data symbols with which said embedded data has been combined 
with said correlation sequence. 

11. An apparatus as claimed in Claim 10, wherein said transform is the 
20 discrete wavelet transform, the data symbols in the transform domain being divided 

into each of a plurality of sub-bands comprising wavelet coefficients, the data being 
added to at least one of the sub-bands. 

12. An apparatus as claimed in any of Claims 5 to 1 1 , wherein each bit of 
25 said PRBS is represented in bipolar form, said data to be embedded modulating the 

symbols of said PRBS by reversing the sign of each bit, said modulated Pseudo 
Random Bit Sequences being combined with respective wavelet coefficients of sgiid 
sub-band. 



30 



13. An apparatus as claimed in any of Claims 10 to 12, comprising 
a pre-processor operable to receive said information material and to generate a 
shifted copy of the received information material, said transform processor being 
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operable to generate a transform domain version of said received information material 
and said shifted copy of said information material, and said correlation processor is 
operable to recover said embedded data by correlating said transform domain data of , 
said received information material and said shifted copy of said information material. 

5 

14. An apparatus as claimed in any preceding Claim, wherein said 
information material is one of audio material, video material and audio/video 
information material. 

10 15. A method for detecting and recovering data embedded in information 

material, said data having been embedded in said information material by modulating a 

predetermined data sequence with the data to be embedded to form modulated data and 

combining said modulated data with said information material, 

said method comprising correlating information material data symbols to which 
15 the modulated data have been added with a correlation data sequence, to form a 

correlation output signal representing the correlation between the information material 

data symbols and said correlation data sequence, and 

recovering said embedded data from said correlation output signal, wherein 

said correlation data sequence comprises a plurality of predetermined data sequence 
20 versions, each version representing a possible form of the predetermined data sequence 

for recovering said embedded data symbols from said modulated data. 

16. A method as claimed in Claim 15, wherein said correlation data 
sequence further comprises a plurality of versions of each of the possible data 

25 sequences shifted with respect to one another. 

17. A method as claimed in Claim 1 5, wherein each of said predetermined 
data sequence versions is a different predetermined data sequence of a set of possible 
predetermined data sequences which may have been used to form said modulated data. 

30 
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18. A computer program providing computer executable instructions, 
which when loaded on to a data processor configures said data processor to operate as 
an apparatus according to any of Claims 1 to 14. 

19. A computer program having computer executable instructions, which 
when loaded on to a data processor causes the data processor to perform the method 
according to Claims 15, 16 or 17. 

20. A computer program product having a computer readable medium 
having recorded thereon information signals representative of the computer program 
claimed in any of Claims 1 8 or 1 9. 

21. An apparatus for detecting and recovering as herein before described 
with reference to the accompanying drawings. 

22. A method of recovering data embedded in information material as 
herein before described with reference to the accompanying drawings. 
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ABSTRACT 

APPARATUS FOR DETECTING AND RECOVERING DATA 

An apparatus for detecting and recovering data embedded in information 
5 material, the data having been embedded in the information material by modulating a 
predetermined data sequence with the data to be embedded to form modulated data and 
combining the modulated data with the information material. The apparatus comprises 
a correlation processor operable in combination with a data sequence processor to 
form a correlation sequence, and a data processor operable under control of the 

10 correlation processor to correlate information material data symbols, with which the 
modulated data have been combined, with the correlation sequence, to form a 
correlation output signal representing the correlation between the information material 
data symbols and the correlation data sequence, and to recover the embedded data 
from the correlation output signal. The correlation sequence comprises a plurality of 

15 predetermined data sequence versions, each version representing a possible form of the 
predetermined data sequence for recovering the embedded data sjonbols from the 
modulated data. The correlation data sequence may have a plurality of predetermined 
data sequence shifted versions, shifted with respect to one another. As a result the 
embedded data may be detectable from the transform domain data symbols in spite of 

20 a relative shift of the transform domain data symbols. Altematively, the 

predetermined data sequence versions may be a set of different predetermined data 
sequences, one of which has been used to generate the modulated data. 

[Figs 8 and 9] 
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